Linear-Quadratic Optimal Controller 

10.3 Optimal Linear Control Systems 

In Chapters 8 and 9 of this book we have designed dynamic 
controllers such that the closed-loop systems display the desired 
transient response and steady state characteristics. The design 
techniques presented in those chapters have sometimes been limited 
to trial and error methods while searching for controllers that meet 
the best given specifications. Furthermore, we have seen that 
in some cases it has been impossible to satisfy all the desired 
specifications, due to contradictory requirements, and to find the 
corresponding controller. 

Controller design can also be done through rigorous mathemat¬ 
ical optimization techniques. One of these, which originated in the 
sixties (Kalman, 1960)—called modern optimal control theory in 
this book—is a time domain technique. During the sixties and sev¬ 
enties, the main contributor to modern optimal control theory was 
Michael Athans, a professor at the Massachusetts Institute of Tech¬ 
nology (Athans and Falb, 1966). Another optimal control theory, 
known as H 0 0 , is a trend of the eighties and nineties. Hoc optimal 
control theory started with the work of Zames (1981). It combines 
both the time and frequency domain optimization techniques to 
give a unified answer, which is optimal from both the time domain 
and frequency domain points of view (Francis, 1987). Similarly to 
Hoc optimal control theory, the so-called Hi optimal control theory 
optimizes systems in both time and frequency domains (Doyle et 
al., 1989, 1992; Saberi et al., 1995) and is the trend of the nineties. 
Since H 0 0 optimal control theory is mathematically quite involved, 
in this section we will present results only for the modern optimal 
linear control theory due to Kalman. It is worth mentioning that 
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very recently a new philosophy has been introduced for system op¬ 
timization based on linear matrix inequalities (Boyd et al., 1994). 

In the context of the modern optimal linear control theory, we 
present results for the deterministic optimal linear regulator prob¬ 
lem, the optimal Kalman filter, and the optimal stochastic lin¬ 
ear regulator. Only the main results are given without deriva¬ 
tions. This is done for both continuous- and discrete-time domains 
with emphasis on the infinite time optimization (steady state) and 
continuous-time problems. In some places, we also present the 
corresponding finite time optimization results. In addition, several 
examples are provided to show how to use MATLAB to solve the 
corresponding optimal linear control theory problems. 


10.3.1 Optimal Deterministic Regulator Problem 

In modern optimal control theory of linear deterministic dynamic 
systems, represented in continuous-time by 

x(£) = A (t)x(t) + B(t)u(t), x(to) = x 0 

we use linear state feedback, that is 


U (t) = — F (t)x(t) 


and optimize the value for the feedback gain, F(t), such that the 
following performance criterion is minimized 


J = min 

u (t) 


t f 


x r (t)Rix(t) + u 1 (t)R 2 u(t) 


T, 


to 


Ri > 0, R 2 > 0 


dt 
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This choice for the performance criterion is quite logical. It re¬ 
quires minimization of the “square” of input, which means, in 
general, minimization of the input energy required to control a 
given system, and minimization of the “square” of the state vari¬ 
ables. Since the state variables—in the case when a linear system 
is obtained through linearization of a nonlinear system—represent 
deviations from the nominal system trajectories, control engineers 
are interested in minimizing the “square” of this difference, i.e. the 
“square” of x(t). In the case when the linear mathematical model 

(10.27) represents the “pure” linear system, the minimization of 
(10.29) can be interpreted as the goal of bringing the system as 
close as possible to the origin (x(t) = 0) while optimizing the 
energy. This regulation to zero can easily be modified (by shifting 
the origin) to regulate state variables to any constant values. 

It is shown in Kalman (1960) that the linear feedback law 

(10.28) produces the global minimum of the performance crite¬ 
rion (10.29). The solution to this optimization problem, obtained 
by using one of two mathematical techniques for dynamic opti¬ 
mization—dynamic programming (Bellman, 1957) and calculus of 
variations—is given in terms of the solution of the famous Riccati 
equation (Bittanti et al., 1991; Lancaster and Rodman, 1995). It 
can be shown (Kalman, 1960; Kirk, 1970; Sage and White, 1977) 
that the required optimal solution for the feedback gain is given by 

F opt (t) = -R.yB r (t)P(t) 

where P(t) is the positive semidefinite solution of the matrix 
differential Riccati equation 

— P (t) = A r (t)P(t) + P(i)A(t) + Ri -P(i)B(i)RJ : 1 B T (t)P(t) 

P (t f ) = 0 


3 



In the case of time invariant systems and for an infinite time 
optimization period, i.e. for tf —» oo, the differential Riccati 
equation becomes an algebraic one 

0 =A r P + PA + Ri - PBR-J^P 


If the original system is both controllable and observable (or only 
stabilizable and detectable) the unique positive definite (semidefi- 
nite) solution of (10.32) exists, such that the closed-loop system 

x(t) = — BR 2 " 1 B T P^x(f), x(to) = x 0 


is asymptotically stable. In addition, the optimal (minimal) value 
of the performance criterion is given by (Kirk, 1970; Kwakernaak 
and Sivan, 1972; Sage and White, 1977) 


1 T 

Jopt = Jmin — -X° Px 0 


Example 10.2: Consider the linear deterministic regulator for 
the F-8 aircraft whose matrices A and B are given in Section 5.7. 
The matrices in the performance criterion together with the system 
initial condition are taken from Teneketzis and Sandell (1977) 


Ri = diag{ 0.01 0 3260 3260}, R 2 = 3260 

x„ = [100 0 0.2 of 

The linear deterministic regulator problem is also known as the 
linear-quadratic optimal control problem since the system is linear 
and the performance criterion is quadratic. The MATLAB function 
lqr and the corresponding instruction 

[F,P,ev]=lqr(A,B, RI, R2) ; 

produce values for optimal gain F, solution of the algebraic Riccati 
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equation P, and closed-loop eigenvalues. These quantities are 
obtained as 


F = 

[-0.004 

0.5557 ■ 

-0.2521 

0.0590] 


" 0.0000 

-0.0016 

-0.0003 

-0.0003 

10 4 

-0.0016 

1.6934 

0.1499 

0.2199 

-0.0003 

0.1499 

0.8211 

-0.0713 


-0.0003 

0.2199 

-0.0713 

0.1361 


Ai, 2 = -0.9631 ± J3.0061, A 3 ,| = -0.0373 ± jO.0837 

The optimal value for the performance criterion can be found from 
(10.34), which produces J op t = 743.9707. 


APPENDIX C. Some Results from Linear Algebra 

Linear algebra plays a very important role in linear system control 
theory and applications (Laub, 1985; Skelton and Iwasaki, 1995). 
Here we review some standard and important linear algebra results. 
Definite Matrices 

Definition C.l: A square matrix M is positive definite if all 
of its eigenvalues have positive real parts, f£e{A z (M)} > 0. It is 
positive semidefinite if f£e{A z (M)} > 0, Mi. In addition, negative 
definite matrices are defined by f£e{A z (M)} < 0, Mi and negative 
semidefinite by i£e{A z (M)} < 0, Mi. 

Null Space 

Definition C.2: The null space of a matrix M of dimensions 
mxnis the space spanned by vectors v that satisfy Av = 0. 
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Systems of Linear Algebraic Equations 

Theorem C.l Consider a consistent (solvable) system of linear 
algebraic equations in n unknowns 

Mx = b (c.l) 

with dim{M} = m x n. Equation (c.l) has a solution if and only 
if (consistency condition) 

rank{[M b]} = rank{M} (c.2) 

In addition, if rank{M} = m, then (c.l) always has a solution. 
For n = m and rank{M} = m the solution obtained is unique. 
Determinant of a Matrix Product 

The following results hold for the determinant of a matrix 
product 

detjMiMo} = det{Mi}det{M;]} (c.3) 

For the proof of the above statement the reader is referred to 
Stewart (1973). This result can be generalized to the product of a 
finite number of matrices. 

Determinant of Matrix Inversion 

By using the rule for the determinant of a product we are able 
to establish the following formula 

det{M " 1} = diilM} (C ' 4) 

This can be proved as follows 

det{MM -1 } = det{I} = 1 = det{M}det{M -1 } 

**(“■'> -5SW 
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Inversion of a Matrix Product 

Consider the problem of finding the matrix inversion to Mi Mo. 
The inversion is a matrix whose product with Mi M 2 produces an 
identity matrix, that is 

Mi M 2 [Inverse] = I (c.5) 

It can be checked that the inverse of the form M 2 " 1 M^ 1 satisfies 
(c.5). This inversion of a product formula can be easily generalized 
to the product of a finite number of matrices. 

Spectral Theorem 

Theorem C.2 IfM. is a symmetric matrix, then its eigenvalues 
are real and M is diagonalizable, i.e. there exists a similarity 
transformation P such that P _1 MP is diagonal. Furthermore, 

1 T 1 

the transformation is unitary, i.e. P = P . 

Proof of this theorem can be found in many standard books 
on linear algebra and matrices (see for example Lancaster and 
Tismenetsky, 1985). 

Integral of a Matrix Exponent 

The following matrix integral formula is useful in some appli¬ 
cations 

T 

J e Mt dt = [e MT - j)m _ 1 (c.6) 

0 

provided that the matrix M is nonsingular. In addition, if all 
eigenvalues of matrix M are asymptotically stable, then 

00 

J e Mt dt = —M -1 
0 
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Vector Derivatives 

The following formulas for vector derivatives are used in Chap¬ 


ter 10. 


. m 

y r My) = My + M r y 




